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Preface 



The Department of Homeland Security (DHS) Office of Inspector General (OIG) was 
established by the Homeland Security Act of 2002 {Public Law 107-296) by amendment 
to the Inspector General Act of 1978. This is one of a series of audit, inspection, and 
special reports prepared as part of our oversight responsibilities to promote economy, 
efficiency, and effectiveness within the department. 

This report addresses the extent to which the Federal Emergency Management Agency 
incorporated corrective actions from Top Officials 4 into National Level Exercise 2009. 
It is based on interviews with employees and officials of relevant agencies and 
institutions, direct observations, and a review of applicable documents. 

The recommendation herein has been developed to the best knowledge available to our 
office, and has been discussed in draft with those responsible for implementation. We 
trust this report will result in more effective, efficient, and economical operations. We 
express our appreciation to all of those who contributed to the preparation of this report. 




Carlton I. Mann 
Assistant Inspector General for Inspections 
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Department of Homeland Security 
Office of Inspector General 

Executive Summary 

National level exercises, formerly designated as Top Officials exercises, 
are designed to reinforce the nation's capability to prepare for, prevent, 
respond to, and recover from large-scale terrorist attacks or natural 
disasters. These exercises test high-level government officials' response 
to simulated attacks and disasters and identify corrective actions resulting 
from problems discovered during these exercises. Our review examined 
how the Federal Emergency Management Agency, which coordinates the 
federal government's role in disaster-related activities, incorporated 
corrective actions identified in Top Officials 4 into National Level 
Exercise 2009. 

The Federal Emergency Management Agency developed a centralized 
Corrective Action Program to address deficiencies identified during 
national level exercises. The agency documented 54 corrective actions in 
Top Officials 4, ten of which were relevant to and incorporated into the 
planning and execution of National Level Exercise 2009. 

While the Federal Emergency Management Agency is responsible for 
inputting information into a corrective action database, tracking these 
corrective actions rests exclusively with participating federal entities. This 
barrier hinders the effectiveness of the Corrective Action Program as the 
Federal Emergency Management Agency has no enforcement authority to 
direct compliance with corrective action implementation. 

We are recommending that the Federal Emergency Management Agency, 
in collaboration with the Exercise and Evaluation Sub-Interagency Policy 
Committee and the National Exercise Program Executive Steering 
Committee, develop reporting regulations for oversight of corrective 
action implementation. The Federal Emergency Management Agency 
agreed with the importance of an oversight system and collaborative 
efforts to include the Exercise and Evaluation Sub-Interagency Policy 
Committee. As an alternative to our recommendation for corrective action 
oversight through federal regulation, we accepted their suggestion to 
develop a mechanism to strengthen corrective action progress in 
collaboration with the White House. 
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Background 



House Report 1 10-862 requires that we review changes made by 
the Department of Homeland Security (DHS) as a result of 
problems identified through Top Officials (TOPOFF) 4, and make 
recommendations for any further improvements needed. 1 To 
satisfy this requirement, we examined the extent to which 
corrective actions identified in TOPOFF 4 were incorporated into 
the planning and execution of DHS' National Level Exercise 
(NLE) 2009. 

The Homeland Security Act (HSA) of 2002 assigned DHS primary 
responsibility to prepare for and sustain the United States' 
capability to reduce vulnerabilities associated with acts of 
terrorism. The DHS Secretary is responsible for coordinating 
federal operations to prepare for, respond to and recover from 
terrorist attacks, major disasters, and other emergencies. 

In February 2003, the President issued Homeland Security 
Presidential Directive (HSPD) 5, Management of Domestic 
Incidents, which identifies steps for improved coordination among 
all levels of government in response to terrorist attacks, major 
disasters and other emergencies. In December 2003, the President 
issued HSPD 8, National Preparedness, which describes how 
federal departments and agencies will prepare for such responses. 
In coordination with other federal departments, agencies, state and 
local governments, and private sector entities, HSPD 8 directed 
DHS to establish a national program to conduct homeland security 
preparedness-related exercises that could be evaluated to determine 
the country's ability to handle man-made or natural disasters. 

The Executive Steering Committee (ESC) coordinates 
departmental and agency roles and responsibilities, and reports to 
the Exercise and Evaluation Sub-Interagency Policy Committee 
(Sub-IPC). The Sub-IPC recommends exercise priorities, goals, 
objectives, schedules, and corrective action issues to the Homeland 
Security Council (HSC) Deputies Committee, which is composed 
of deputy cabinet secretaries of the relevant federal agencies. 

Within DHS, the Federal Emergency Management Agency 
(FEMA) coordinates the federal government's role in all domestic 
disaster- related activities. In April 2007, FEMA established the 



1 Top Officials exercises were designated as National Level Exercises beginning in Fiscal Year 2009. 
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National Preparedness Directorate (NPD) to oversee the 
coordination and development of strategies necessary to prepare 
for terrorist incidents and natural disasters. As part of this mission, 
NPD provides policy and planning guidance that builds prevention, 
protection, response, and recovery capabilities among all levels of 
federal, state and local governments. 

Within NPD, the National Exercise Division (NED) established the 
National Exercise Program (NEP) to improve the delivery and 
organization involved with planning for, developing and executing 
preparedness-related national level exercises for the federal 
government. NEP is the primary tool for assessing the country's 
ability to prevent, protect, respond to and recover from 
catastrophic events. National level exercises are conducted to test 
and identify planning and resource gaps in existing strategies, 
policies and procedures. 

A White House memorandum entitled, "National Exercise 
Program," identified by many observers as the NEP charter, 
requires a corrective action process to ensure lessons from 
exercises are either sustained or improved as appropriate. To 
facilitate a corrective action process, FEMA developed a 
Corrective Action Program (CAP) to address preparedness 
deficiencies identified during exercises, policy discussions, and 
real- world events and refers issues to departments and agencies for 
priority and remedial action. Specifically, the CAP identifies, 
assigns, and tracks corrective actions. FEMA's NPD serves as the 
responsible agent for the CAP, and provides administrative and 
technical support to other federal departments and agencies. 

National level exercises have four homeland security mission 
areas: prevent, protect, respond, and recover. 

• Prevent Mission: Actions taken to avoid an incident or 
intervene to stop an incident from occurring. 

• Protect Mission: Reduce the likelihood of attack on assets or 
systems and limit the impact should an attack occur. 

• Respond Mission: Implement immediate actions to save lives, 
protect property, and meet basic human needs. 



After TOPOFF 4 and NLE 09 exercises, a fifth mission area - mitigation - was added to the homeland 
security mission areas. Mitigation is the effort to reduce loss of life and property by lessening the impact of 
disasters. 



FEMA's Implementation of Recommendations from Top Officials 4 

Page 3 



• Recover Mission: Develop, coordinate, and execute service- 
and site-restoration plans and reconstitute government 
operations and services through individual, private- sector, 
nongovernmental, and public assistance programs. 

National level exercises are categorized into four tiers. These tiers 
reflect the priority for federal participation, with Tier I as the 
highest and Tier IV as the lowest, and are assigned based on their 
alignment with federal government- wide strategic and policy 
priorities. 

As stated in the NEP Implementation Plan, a national level 
exercise is the single annual operations-based NEP Tier I exercise 
focused on White House-directed, U.S. government- wide strategy 
and policy-related issues that require the participation of all 
appropriate department and agency principals. There are 5 NEP 
Tier I exercises annually. TOPOFF 4 and NLE 2009 were both 
Tier I exercises. Tier II exercises are focused on strategy and 
policy issues supported by all appropriate departments and 
agencies. Tier III exercises focus on operational, tactical, or 
organization-specific objectives not requiring broad interagency 
headquarters-level involvement to achieve objectives. Tier IV 
exercises include state, territorial, local, or tribal governments and 
private sector entities as the primary training audience or subject of 
evaluation. 



Results of Review 

All TOPOFF 4 Corrective Actions Were Not Relevant for NLE 
2009 

As part of the National Exercise Program (NEP), emergency preparedness 
exercises attempt to build an integrated, interagency federal, state, 
territorial, local, and private sector capability to prevent terrorist attacks, 
and respond to and recover from any terrorist attack or major disaster. 
The first in a series of national preparedness exercises, TOPOFF 2000, 
simulated biological attacks in Colorado and New Hampshire. TOPOFF 2 
simulated a radiological dispersal device detonation and an outbreak of 
pneumonic plague. The exercise included pre-exercise intelligence play, a 
cyber- attack, and credible terrorism threats against other locations. 
TOPOFF 3 was a large-scale, multipoint terrorist attack using biological 
and chemical weapons. TOPOFF 4 simulated a radiological dispersal 
device detonation near a power plant. 
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During the planning and conduct of NLE 2009, interagency planners were 
also engaged in planning NLE 2010, NLE 201 1, and multiple Tier I and 
Tier II exercises. With these multiple events and associated planning 
meetings, concerns were expressed regarding the viability planning, 
evaluating, and incorporating lessons learned in the annual national level 
exercise. 

FEMA officials stated that for NLE 2009, planners recognized that 
TOPOFF 4 and previous national level exercises focused almost 
exclusively on response operations and law enforcement, with insufficient 
emphasis placed on prevention and intelligence activities. Therefore, 
incorporating corrective actions into subsequent exercises may not be 
relevant in most cases since they do not want to test the same capabilities 
each year. FEMA officials concluded that it would be more beneficial to 
alternate the type of threat each year to incorporate pre- and post-incident 
exercises. As a result, this would better ensure that each of the homeland 
security mission areas was tested over time. 

In preparation for NLE 2009, exercise planners agreed that a primary area 
of concern should be securing the southwest border and preventing 
terrorists from entering and attacking the United States. Because of these 
priorities, NLE 2009 focused exclusively on terrorism prevention and 
protection, as opposed to incident response and recovery, which resulted 
in different issues and objectives between TOPOFF 4 and NLE 2009. 
Exercise planners concluded that NLE 2009, as a prevention-focused 
exercise, would continue to accomplish national preparedness goals and 
objectives because of the emphasis placed on intelligence and information 
sharing that occurs in all national level exercises. 

NLE 2009 was the first major exercise exclusively dedicated to terrorism 
prevention and protection. Specifically, this exercise focused on 
preventing an adversary's entry into the country through air, border, and 
maritime security and protecting various infrastructures and information 
through various alert and notification systems. Unlike past national level 
exercises, NLE 2009 did not have an incident response and recovery 
component, but highlighted intelligence and information sharing. 

During the planning process for NLE 2009, FEMA officials reviewed past 
exercise improvement plans to determine (1) specific deficiencies 
requiring further examination, and (2) compatibility with the overall 
terrorism prevention focus for NLE 2009. TOPOFF 4 exercise simulated 
a bomb explosion that involved responding to radiological dispersal 
device attacks. The TOPOFF 4 Improvement Plan identified 54 corrective 
actions, of which 12 pertained to radiological events, 19 pertained to 
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recovery, 12 pertained to response, one was considered outside of 
TOPOFF 4's scope, and ten pertained to terrorism and prevention 
protection. 

The TOPOFF 4 corrective actions incorporated into NLE 2009 were 
identified during the exercise planning phase as actions to be validated 
during the functional prevention exercise. The corrective actions include 
Emergency Operations Center management, on-site incident management, 
and public information and warning. The TOPOFF 4 corrective actions 
incorporated into NLE 2009 are listed below. 

1. Incorporate national scenario-based guidance into regional planning, 
training, and exercise programs such as the Regional Interagency 
Steering Committee (RISC) or the Regional Assistance Committee 
(RAC). 

2. Consider the use of virtual tools (such as web conferencing and chat 
rooms) to supplement National Incident Communications Conference 
Line (NICCL) calls. 

3. Evaluate smart practices and suggestions on information management 
identified in the After Action Report (AAR). 

4. Investigate information technology solutions that support e-mail 
distribution lists that can be easily modified. 

5. Consider mechanisms to promote cross-jurisdictional coordination by 
public affairs officials. 

6. Develop and promulgate written Strategic Communication Planning 
guidance; establish and exercise interagency strategic communication 
team to address national themes, effects, tasks, international 
engagement strategy, and processes and procedures. 

7. Review and align meeting and reporting schedules. 

8. Consider scope, attendance, and classification level of senior 
leadership meetings, as well as procedures for capturing and 
disseminating discussions, decisions, and taskings. 

9. Review and update policy documents to clarify the purpose, roles, and 
responsibilities for various private sector nongovernmental 
organizations (NGO). 
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10. Articulate and institutionalize a process for private sector and NGO 
engagement in national-level exercises, including authority for 
planning, programming, and budgeting for national and venue working 
groups. 

FEMA officials provided the following rationale for TOPOFF 4 corrective 
actions that were incorporated in NLE 2009: 

Corrective Action 1 

This corrective action relates to the On-Site Incident Management 
capability. This capability provides the means to accomplish a mission 
and achieve desired outcomes by performing critical tasks, under specified 
conditions, to target levels of performance. 

Corrective Actions 2, 3, 4, 5 and 6 

Although the focus of NLE 2009 was intelligence and information 
sharing, the exercise provided an opportunity to validate TOPOFF 4 
corrective actions related to Public Information and Warning. 
Specifically, the exercise included interagency communication protocols 
to ensure that accurate and credible law enforcement and intelligence 
information was provided to the public. Electronic message transmissions 
and scheduled teleconferences among national, state, local and private 
sector participants facilitated information exchange. In addition, virtual 
tools, such as web conferencing and chat rooms, were a primary source of 
communication throughout the exercise. 

Corrective Actions 7 and 8 

These corrective actions relate to the Emergency Operations Center 
Management capability, and were incorporated in both the planning and 
conduct of NLE 2009. The DHS Secretary, White House staff, and other 
senior federal leaders participated in the exercise and related discussions. 

Corrective Actions 9 and 10 

These corrective actions relate to the Emergency Operations Center 
Management capability. Non-government agencies, as well as critical 
infrastructure facilities, participated in NLE 2009, with the National Joint 
Information Center serving as a focal point for coordinated and timely 
release of incident-related information to the public and media. 

All TOPOFF 4 corrective actions were not incorporated into NLE 2009 
since FEMA officials concluded that it is beneficial to alternate the type of 
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threats and related capabilities that are exercised. However, FEMA 
officials said that other federal departments and agencies have planned or 
conducted exercises related to corrective action scenarios from TOPOFF 
4, as described in the following examples: 

• The Department of Energy sponsored Empire 09, a three phase 
exercise which focused on a radiological dispersal device detonation in 
an urban environment. This provided an opportunity for response 
agencies to focus on Federal Radiological Monitoring and Assessment 
Center integration, responsibilities, and reporting. 

• The Environmental Protection Agency sponsored Liberty RadEX 
2010, which focused on long-term recovery from a radiological 
dispersal device detonation. This exercise provided an opportunity for 
participants to focus on recovery issues. 

Barriers Exist That Hinder the Effectiveness of a Corrective 
Action Program 

A major component of exercise evaluation is the After Action Conference 
which provides an opportunity for entities that participated in an exercise 
to exchange feedback and make necessary changes to an After Action 
Report/Improvement Plan. As part of the After Action Conference, 
corrective actions are assigned to a responsible person or agency. 

The NEP Implementation Plan requires the establishment of a corrective 
action program, administered by DHS, to provide a government- wide 
process for identifying, assigning, and tracking remediation of interagency 
issues identified through exercises. Specifically, each participating 
department and agency is required to maintain a system for (1) identifying 
issues to be remedied, (2) making assignments to resolve the issues, and 
(3) tracking progress on issue resolution. The NEP Implementation Plan 
further requires that departments and agencies identify points of contact 
(POC) to oversee the implementation of corrective action plans, and 
update the CAP System on progress made towards implementing 
corrective actions. 

FEMA's CAP System enables federal entities to access and track 
improvement plans from a single location, and ensures that lessons from 
exercises are either sustained or improved, as appropriate. However, we 
identified challenges that reduce the effectiveness of the CAP process. 
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Even though each participating agency or department is responsible for 
tracking and implementing corrective actions, we determined that not all 
agencies and departments that participated in national level exercises have 
assigned a POC to oversee corrective action implementation, and update 
the CAP System. FEMA managers explained that more emphasis was 
placed on the actual operation of the exercises, and less emphasis on 
corrective actions needed to resolve consequence management issues 
identified during the exercises. FEMA officials added that even when a 
POC is assigned to monitor corrective actions, if that person leaves the 
department or agency, a replacement POC is often not assigned to follow 
through on corrective actions. 

FEMA is responsible for inputting initial corrective action data from 
national level exercises into the CAP System. However, tracking and 
implementing these corrective actions rests exclusively within the 
discretion of each participating agency or department, with FEMA having 
no enforcement authority. In addition, we identified no explicit statutory 
or executive authority that directs federal agencies to comply with 
corrective action implementation requirements outlined under the NEP 
Implementation Plan. 

A common framework for corrective action follow-up is necessary to 
ensure lessons from exercises are either sustained or improved, as 
appropriate. In the absence of statutory or executive requirements for 
participation in a corrective action process by all national level exercise 
participants, there is reduced assurance that needed improvements 
identified during exercises will be monitored, tracked, and implemented, 
as appropriate. 

Recommendation 

We recommend that the Director, Office of Policy and Program 
Analysis: 

Recommendation #1 ; Develop and implement reporting 
regulations for federal departments and agencies to assign POCs to 
oversee corrective action implementation, and update the CAP 
System on a consistent basis. This may require collaboration with 
the Exercise and Evaluation Sub-Interagency Policy Committee 
and the NEP Executive Steering Committee. 
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Management Comments and OIG Analysis 

We evaluated FEMA's written comments and have made changes to 
the report where we deemed appropriate. Below is FEMA's written 
response to our recommendation and our analysis of the response. A 
copy of FEMA's response, in its entirety, appears in Appendix B. 

FEMA Response: With regard to the recommendation, we concur 
with the importance of an oversight system and collaborative 
efforts to include the Exercise and Evaluation Sub-Interagency 
Policy Committee. We do not concur that development and 
implementation should be promulgated through a federal 
regulation, but suggest that FEMA work collaboratively with the 
White House to develop a mechanism, such as a White House 
Directive, to strengthen the process of the corrective action 
progress to include (1) making assignments to resolve issues, (2) 
tracking progress on issues resolution, and (3) identifying POCs to 
oversee the implementation of corrective action plans. As a side 
note, we are making strides regarding incorporating corrective 
actions into National Exercise Program objectives. We believe the 
revision of the National Exercise Program (currently underway) 
will standardize exercise doctrine and the corrective action process 
and strengthen the good conduct of departments and agencies 
participating in the National Exercise Program. 

OIG Evaluation: As an alternative to our recommendation for 
corrective action oversight through federal regulation, we accept 
FEMA's suggestion to collaborate with the White House in 
developing a mechanism to strengthen corrective action progress. 
The recommendation will remain unresolved and open pending 
our receipt and review of the specific mechanism that will be 
developed to satisfy the intent of this recommendation. 
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Appendix A 

Purpose, Scope, and Methodology 



House Report 110-862, Department of Homeland Security 
Appropriations Bill, 2009, requires that we review changes made 
by the Federal Emergency Management Agency as a result of 
problems identified through TOPOFF 4, and make 
recommendations for any further improvements that may be 
needed. Pursuant to this requirement, we determined how the 
corrective actions identified in the TOPOFF 4 exercise series were 
incorporated into the planning and execution of NLE 2009. 

We observed NLE 2009 activities from July 27 through July 31, 
2009, and attended the After Action Conference in February 2010. 
An overview of NLE 2009 results were incorporated into an After 
Action Report. FEMA officials provided us with a draft of this 
report in February 2010. As such, we continued our fieldwork 
from March to May 2010, and interviewed FEMA top officials 
responsible for the design and execution of national level 
exercises. We analyzed relevant laws, policy documents, and 
records concerning the evolution and ongoing priorities of these 
exercises. 

This review was conducted under the authority of the Inspector 
General Act of 1978, as amended, and according to the Quality 
Standards for Inspections, issued by the Council of the Inspectors 
General on Integrity and Efficiency. 



FEMA's Implementation of Recommendations from Top Officials 4 

Page 11 



Appendix B 

Management Comments to the Draft Report 
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MEMORANDUM FOR: 



Carlton I. Mann 

Assistant Inspector General - Inspections 
Office of Inspector General 



FROM: 



David J. Kau 
Director 

Office of Policy and Program \n.iIyMs 
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Comments on OIG Draft Report. FSMA s Implementation 
of Recommendations from Top Officials 4 



Thank you for the opportunity to review and comment on the Office of Inspector 
General's (OIG's) subject draft audit report. As the Federal Emergency Management 
Agency (FEMA) works toward refining its programs, the OIG's independent analysis of 
program performance greatly benefits our ahility to continuously improve our activities. 

We find the report properly identities barriers that hinder the effectiveness of the current 
Corrective Action Program with regard to National Level Exercises (NLEs). In addition 
to providing comments on the findings and recommendation, wc would like to clarily one 
area regarding the corrective actions related to NLE 09. The 1 corrective actions were 
not radiological incident response actions as indicated in the report, but rather these 
corrective actions were identified during the NLE 09 planning phase as actions to be 
validated during the functional prevention exercise. These include Emergency Operations 
Center (EOC) management, on-site incident mnnagemenl. and public information and 
warning, In addition, we find the subtitle of" the Results in Review section does not 
accurately reflect the summary contents and recommend the suhlillc be changed from All 
TOPOI-F 4 Corrective Actions Were Not Incorporated into NUZ 200V to All TOPOl-F 4 
Corrective Actions Were Not Relevant for NLE 2009, Finally, we would recommend a 
footnote be added to the section pertaining to homeland security mission areas {page 2) to 
indicate that since TOPOFF 4 and NLE O'J. a fifth mission area obligation has been 
added. 
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With regard to the recommendation, we concur with the importance of an oversight 
system and collaborative efforts to include the Exercise and Evaluation Sub-lnteragency 
Policy Committee. We do not concur that development and implementation should be 
promulgated through a federal regulation, but suggest that FEMA work collaboratively 
with the White House to develop a mechanism, such as a White House Directive, to 
strengthen the process of the corrective action progress to include (1) making 
assignments to resolve issues, (2) tracking progress on issues resolution, and [3) 
identifying points of contacts (POCs) to oversee the implementation of corrective action 
plans. As a side note, we are making strides regarding incorporating corrective actions 
into National Exercise Program objectives. We believe the revision of the National 
Exercise Program, which is currently underway, will standardize exercise doctrine and 
the corrective action process and strengthen the good conduct of department and agencies 
participating in the National Exercise Program. 

Thank you again for the opportunity to comment on this draft report and we look forward 
to working with you on other issues as we both strive to improve FEMA. 
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Appendix C 
Major Contributors 



Deborah Outten-Mills, Chief Inspector 
Jacqueline Simms, Senior Inspector 
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ADDITIONAL INFORMATION AND COPIES 

To obtain additional copies of this report, please call the Office of Inspector General (OIG) at (202) 254-4100, 
fax your request to (202) 254-4305, or visit the OIG web site at www.dhs.gov/oig. 



OIG HOTLINE 

To report alleged fraud, waste, abuse or mismanagement, or any other kind of criminal or noncriminal 
misconduct relative to department programs or operations: 

• Call our Hotline at 1-800-323-8603; 

• Fax the complaint directly to us at (202) 254-4292; 

• Email us at DHSOIGHOTLINE@dhs.gov; or 

• Write to us at: 

DHS Office of Inspector General/MAIL STOP 2600, 
Attention: Office of Investigations - Hotline, 
245 Murray Drive, SW, Building 410, 
Washington, DC 20528. 



The OIG seeks to protect the identity of each writer and caller. 



